Z2> f 



PATENT ABSTRACTS OF JAPAN 
(1 1 publication number : 2001-1 2451 7 
(43)Date of publication of application : 1 1 .05.2001 



(51)Int.CI. G01B 11/00 
G01B 11/24 
G06F 17/10 
G06T 3/00 



(21 Application number : 1 1-306076 (71)Applicant : MITSUTOYO CORP 

(22)Date of filing : 27.1 0.1 999 (72)Inventor : FU JIMOTO IKUMATSU 
OKAMOTO KIYOKAZU 



(54) DISSOLUTION METHOD FOR GEOMETRIC STRAIN IN IMAGING SPACE 
(57)Abstract: 

PROBLEM TO BE SOLVED: To provide an easily dissolving geometric strain in 
imaging space to be within a prescribed tolerable error value. 

SOLUTION: Affine transformation series, image pickup figure center of image pickup 
figure on the figure center of reference figure, is determined from a reference space 
set by a plurality of reference figures and an imaging space obtained by imaging it. In 
the case the position error after transformation does no fit in a tolerable error value, 
the reference space is split and affine transformation and space separation are 
repeated, until the evaluation value of strain on the separated space. 
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CLAIMS 

[Claim(s)] 

[Claim 1] It is the approach of canceling the geometric distortion of the image pick-up 
space which picturizes measured space and is obtained. The reference space which 
has two or more reference graphic forms which were installed in the location 
equivalent to measured space, and which the coordinate value location of **** of 
each reference graphic form and the reference original-drawing heart of the whole 
reference graphic form understands beforehand is prepared. The imprint 
original-drawing heart which is **** of two or more whole image pick-up graphic 
forms corresponding to said two or more reference graphic forms in the image pick-up 
space which picturizes said measured space and is obtained Put on the reference 
original-drawing heart of said reference space, and to reference Fig. **** which is 
**** of two or more of said reference graphic forms, respectively The first affine 
transformation which copies imprint Fig. **** which is **** of two or more of said 
image pick-up graphic forms is performed. Perform space division which divides said 



reference space into two or more first division space, and the position error of each 
imprint Fig. heart after said first affine transformation and the reference Fig. heart 
corresponding to it is searched for as a distortion evaluation value about said each 
first division space. Said first division space is classified into the division space where 
said distortion evaluation value is settled in a predetermined allowable-error value, 
and the division space in which it does not fit. As opposed to the division space which 
is not settled in said allowable-error value to reference Fig. **** of the interior As 
opposed to the first division space which performs the second affine transformation 
which copies corresponding imprint Fig. **** of image pick-up space, and is not 
settled in said allowable-error value after first [ said ] and the second affine 
transformation The geometric distortion dissolution approach of the image pick-up 
space which performs space division divided into two or more second division space, 
and is hereafter characterized by repeating affine transformation and space division 
until said distortion evaluation value is settled in said allowable-error value about the 
division space which is not settled in said allowable-error value. 

[Claim 2] Said distortion evaluation value is the geometric distortion dissolution 
approach of the image pick-up space according to claim 1 characterized by 
considering as the maximum of the norm of the position error of said reference Fig. 
heart and imprint Fig. heart after said affine transformation. 

[Claim 3] Said distortion evaluation value is the geometric distortion dissolution 
approach of the image pick-up space according to claim 1 characterized by asking 
said reference Fig. heart for the nearby imprint Fig. heart, and considering as the 
maximum of the norm of a position error with the imprint Fig. heart corresponding to 
the nearby imprint Fig. heart and said reference Fig. heart. 

[Claim 4] The geometric distortion dissolution approach of the image pick-up space 
according to claim 1 characterized by determining an affine transformation child with 
the least square method over the position error of the location after conversion, and 
the reference Fig. heart corresponding to this in said each affine transformation. 
[Claim 5] Said each space division is the geometric distortion dissolution approach of 
the image pick-up space according to claim 1 characterized by every direction 
considering as abbreviation 1 / quadrisection of space depended comparatively for 2 
minutes. 

[Claim 6] Said two or more reference graphic forms are the geometric distortion 
dissolution approaches of the image pick-up space according to claim 1 characterized 
by being arranged on one boundary of each division space when a part of the 
reference Fig. heart divides said reference space or said reference space. 
[Claim 7] The geometric distortion dissolution approach of image pick-up space 
according to claim 1 that one boundary of each reference field which divides said 
reference space or said reference space, and is obtained is characterized by drawing a 
rectangle, a triangle, or radii. 



DETAILED DESCRIPTION 



[Detailed Description of the Invention] 
[0001] 

[Field of the Invention] This invention relates to the approach of canceling geometric 
distortion produced to image pick-up space, when the image which picturized 
measured space with the CCD camera etc. and was obtained performs flat-surface 
configuration measurement. 
[0002] 

[Description of the Prior Art] Conventionally, the step of the investigation to the 
cause of distortion, such as an approach of performing approximation by ** dividing 
polynomial as reference space using a grid-like graphic form as an approach of 
canceling geometric distortion produced to image pick-up space, a method of 
performing ** whole space-affine transformation, and ** lens distortion, is stepped on, 
and the approach of carrying out the approximation of function etc. is used. 
[0003] 

[Problem(s) to be Solved by the Invention] However, as for this, count cost becomes 
high, although high degree of accuracy is expectable by the approach of performing 
approximation by the dividing polynomial of ** if a high order polynomial is used. ** 
The method of performing whole space-affine transformation cannot respond to a 
nonlinear distortion, and cannot expect high degree of accuracy. ** The approach of 
carrying out the approximation of function is not easy to ask for a function, and high 
degree of accuracy cannot be expected to distortion from which curvature changes a 
lot locally. Therefore, there was no practical approach of canceling distortion of image 
pick-up space easily within the assignment allowable error of arbitration 
conventionally. 

[0004] This invention aims at offering the approach of canceling easily the geometric 
distortion of image pick-up space within a predetermined allowable-error value. 
[0005] 

[Means for Solving the Problem] This invention is the approach of canceling the 
geometric distortion of the image pick-up space which picturizes measured space and 
is obtained. The reference space which has two or more reference graphic forms 
which were installed in the location equivalent to measured space, and which the 
coordinate value location of **** of each reference graphic form and the reference 
original-drawing heart of the whole reference graphic form understands beforehand is 
prepared. The imprint original-drawing heart which is **** of two or more whole image 
pick-up graphic forms corresponding to said two or more reference graphic forms in 



the image pick-up space which picturizes said measured space and is obtained Put on 
the reference original-drawing heart of said reference space, and to reference Fig. 
**** which is **** of two or more of said reference graphic forms, respectively The 
first affine transformation which copies imprint Fig. **** which is **** of two or more 
of said image pick-up graphic forms is performed. Perform space division which 
divides said reference space into two or more first division space, and the position 
error of each imprint Fig. heart after said first affine transformation and the reference 
Fig. heart corresponding to it is searched for as a distortion evaluation value about 
said each first division space. Said first division space is classified into the division 
space where said distortion evaluation value is settled in a predetermined 
allowable-error value, and the division space in which it does not fit. As opposed to 
the division space which is not settled in said allowable-error value to reference Fig. 
**** of the interior The second affine transformation which copies the location after 
the first affine transformation of corresponding imprint Fig. **** of image pick-up 
space is performed. As opposed to the first division space which is not settled in said 
allowable-error value after first [ said ] and the second affine transformation Space 
division divided into two or more second division space is performed, and it is 
characterized by repeating affine transformation and space division until said 
distortion evaluation value is hereafter settled in said allowable-error value about the 
division space which is not settled in said allowable-error value. 

[0006] According to this invention, the geometric distortion of the image pick-up 
space which picturizes measured space with a CCD camera etc. and is obtained is 
easily cancelable with the repeat of affine transformation and space division. 
[0007] Let said distortion evaluation value preferably be the maximum of the norm of 
the position error of said reference Fig. heart and imprint Fig. heart after said affine 
transformation in this invention. Or said distortion evaluation value can ask said 
reference Fig. heart for the nearby imprint Fig. heart, and can also make it the 
maximum of the norm of a position error with the imprint Fig. heart corresponding to 
the nearby imprint Fig. heart and said reference Fig. heart again. Moreover, in each 
affine transformation in this invention, an affine transformation child is determined 
with the least square method over the position error of the location after conversion, 
and the reference Fig. heart corresponding to this. Furthermore, in this invention, 
every direction considers space division as abbreviation 1 / quadrisection of space 
depended comparatively for 2 minutes. Moreover, in this invention, two or more 
reference graphic forms are arranged on one boundary of each division space when a 
part of that reference Fig. heart divides said reference space or said reference space, 
for example. Furthermore, in this invention, one boundary of each reference field 
which divides reference space or reference space and is obtained shall draw a 
rectangle, a triangle, or radii. 
[0008] 



[Embodiment of the Invention] Hereafter, this invention is explained with reference to 
a drawing. In advance of the gestalt of concrete operation, the algorithm of a 
geometric distortion dissolution of the image pick-up space by this invention is 
explained in order of space division ** for storing all position errors in an 
allowable-error value based on (1) reference space, image pick-up space, the decision 
of (2) affine transformation types and a distortion evaluation value, and (3) distortion 
evaluation value. 

[0009] (1) As shown in reference space and image pick-up space drawing 1 , prepare 
the reference space 4 where two or more reference graphic forms 1 were set up. Each 
reference graphic form 1 has the reference Fig. heart 2 (bk (k= 1 , 2, — , n)) which is 
**** for which accuracy understands the coordinate location beforehand. Moreover, 
the heart of the reference graphic form 1 whole shall be the original-drawing heart 3 
(bg), and shall understand the coordinate value also for this beforehand. On the other 
hand, the graphic form corresponding to the reference graphic form 1 is used as the 
image pick-up graphic form 5 about the image pick-up space 8 which picturized the 
reference graphic form 1 by CCD, use the heart corresponding to the reference Fig. 
heart 2 as the imprint Fig. heart 6 (ak (k= 1, 2, — , n)), and let the heart of the whole 
image pick-up graphic form corresponding to the reference original-drawing heart 3 
be the imprint original-drawing heart 7 (ag). In addition, each reference graphic form 1 
of the reference space 4 is arranged on the boundary of division space when a part of 
the reference Fig. heart 2 performs space division explained later. 
[0010] In the relation between such reference space 4 and the image pick-up space 8, 
as shown in drawing 2 , the original-drawing heart 3 (bg) and the imprint 
original-drawing heart 7 (ag) are piled up, and affine transformation which copies the 
imprint Fig. heart 6 to the reference Fig. heart 2 is performed. At this time, it verifies 
whether this distortion evaluation value epsilon is settled in the predetermined 
allowable-error value delta considering the maximum of the position error of the 
location and the corresponding reference Fig. heart 2 after affine transformation as a 
distortion evaluation value epsilon to all the imprint Fig. hearts 6. Here, in the minimum 
space (square field surrounded by four adjacent points) when doing n division of the 
reference space 4 at the shape of a cross grating, when affine transformation is 
performed, it is assumed that a group of the reference Fig. heart 2 and a group of the 
imprint Fig. heart 6 which are surely settled in the appointed allowable-error value are 
chosen beforehand. 

[001 1] (2) Set the reference Fig. heart to an affine transformation type and distortion 
evaluation value epsilonn piece imprint Fig. heart ak=(akx, aky) T to bk=(bkx, bky) T, 
and make an affine transformation child into the one following. 
[0012] 
[Equation 1] 



A = 



r ^ 
x 1 x 2 



x 3 x 4 



[0013] Here, it is assumed that the coordinate value of each **** is changed into the 
coordinate value of the system of coordinates which make a zero the imprint 
original-drawing heart ag and the original-drawing heart bg, respectively. To each k, 
bk-Aak is considered and Function (x1, x2, x3, x4) f is set with the two following. 
[0014] 
[Equation 2] 



[0015] And the affine transformation child A is determined with the least square 
method. That is, from delta f/delta x1=delta f/delta x2=delta f/delta x3=delta f/delta 
x4=0, a linear equation with the three following is solved and it asks for the affine 



transformation child A. 

[0016] 

[Equation 3] 




[0017] It will become the four following if several 3 is solved. 

[0018] 

[Equation 4] 
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[0019] At this time, the affine transformation child A in image pick-up space sets the 
coordinate value of image pick-up space to (X, Y), and the coordinate value of 
reference space is expressed with the five following as (x, y). 
[0020] 
[Equation 5] 
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[0021] As mentioned above, let the amount of position errors of the reference Fig. 
heart 2 and the imprint Fig. heart 6 after conversion be the distortion evaluation value 
epsilon. For example, the six following define the distortion evaluation value epsilon of 
the image pick-up space S as maximum of the norm of the position error of the 
reference Fig. heart and the imprint Fig. heart after affine transformation. 
[0022] 
[Equation 6] 

Epsilon=sup||Aa-b||a**S [0023] Here, norm ||-|| is defined as follows. 

[0024] 

[Equation 7] 
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[0025] Moreover, let b be the reference Fig. heart of the reference space 
corresponding to imprint Fig. heart a**S. In addition, in other examples of the 
distortion evaluation value epsilon, the following definitions are also considered by 
using a as the imprint Fig. heart like the above, using b as the reference Fig. heart. 
That is, a and x which becomes x**S are the cases where consider a nearby imprint 
location to the reference Fig. heart which makes ||Ax-b|| min, and it is defined as it as 
follows as the imprint Fig. heart of the point corresponding to this nearby imprint 
location and reference Fig. heart, and maximum of a position error. 
[0026] 
[Equation 8] 
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[0027] However, an argument here shall use the distortion evaluation value epsilon 
shown in several 6. If it does not judge and enter [ whether this distortion evaluation 
value epsilon enters in the predetermined allowable-error value delta, and ], image 
pick-up space (reference space) is divided, and the next affine transformation is 
performed to the divided image pick-up space. Hereafter, a repeat is performed for 
the same processing until the distortion evaluation value epsilon enters in the 
predetermined allowable-error value delta, and geometric distortion is canceled. 
[0028] (3) — the space division for storing a position error epsilon in the 
allowable-error value delta — as mentioned above, the reference Fig. heart to n 
imprint Fig. heart ak=(akx, aky) T is set to bk=(bkx, bky) T, and an affine 
transformation child is expressed with several 1, and the location of the 
original-drawing heart [ in / for the imprint Fig. heart of image pick-up space / bg and 
reference space ] is set with ag, and the step of space division is as follows. 
[0029] (The 1 st step) As shown in drawing 1 , it asks for the first affine transformation 
child AO by the approach mentioned above from reference Fig. heart 2:bk= (bkx, bky) 
and corresponding imprint Fig. heart 6:ak= of the whole space S of the reference 
space 4 (akx, aky). 

[0030] (The 2nd step) As shown in drawing 3 A, let division space by the first division 
of the reference space 4 be S1, S2, S3, and S4. In this example, space division is 
quadrisection of the space where every direction is divided abbreviation 1 / 2, and 
becomes, as a division boundary line passes along the reference Fig. heart 2 top. At 
this time, a part of reference Fig. heart 2 is on the boundary of each division space (a 
reference field or reference space), the boundary of each reference field draws a 
rectangle (square or or rectangle), and that outermost frame constitutes the 
reference space 4. However, a setup of reference space on which the boundary of 



each reference field in reference space or reference space draws radii or a triangle is 
also possible. The division space of the image pick-up space 8 corresponding to the 
first division space S1 of this reference space 4 - S4 is SV-S4" as shown in drawing 3 
B. Since such division space corresponds to one to one, paying attention to the 
division space of reference space, it explains below. 

[0031] such division space is alike, respectively, it receives and the primary maximum 
of the position error of the imprint Fig. heart after conversion and the reference Fig. 
heart corresponding to this is classified into the division space settled in the 
predetermined allowable-error value delta in division space, and the division space in 
which it does not fit as distortion evaluation value epsilonk (k= 1, 2, 3, 4). Since the 
distortion evaluation value epsilon is settled in the allowable-error value delta in the 
division space S1 among the space S1 - S4 which were quadrisected in the example 
of drawing 3, distortion amendment is possible at the first afflne transformation child 
AO. That is, in the division space SI. the further division does not have the need and 
the amendment to which a position error is settled in the assignment allowable-error 
value delta is attained by the following affine transformation type. 
[0032] 
[Equation 9] 



= A 



[0033] The distortion evaluation value epsilon progresses to the following step about 
the remaining division space S2 and S3 and S4 which are not settled in the 
allowable-error value delta. 

(The 3rd step) It is already changed by the first affine transformation child AO also 
about such division space S2 and S3 and S4. At this time, not a but AOa is used for 
the imprint Fig. heart of the imprint Fig. heart by the decision of an affine 
transformation type, and evaluation of distortion. In the following arguments, the 
division space S2 is taken for an example. That is, in the decision of an affine 
transformation type, Function (x1, x2, x3, x4) f is set with the ten following instead of 
bk-Aak using bk-A (AOak), and the affine transformation child A is determined with 
the least square method. 
[0034] 

[Equation 10] 



[0035] Moreover, the distortion evaluation value epsilon is calculated by the 11 

following. 

[0036] 

[Equation 1 1] 

Epsilon=sup||A (A0a)-b||a**S2 [0037] In this way, when the obtained affine 
transformation child A is set to A1 , affine transformation child A (2) in the division 
space S2 is expressed with a degree type. 
[0038] 

[Equation 12] 
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[0039] Therefore, A(2) =A1A0 is materialized. This affine transformation child A (2) is 
composition of the first affine transformation child AO in the image pick-up whole 
space, and two transformation quantities of the second affine transformation child A1 
in the first-divided division space S2. Here, as shown in drawing 3 A and B, it is the 
second division (also in this case) further about the division space S2. Every direction 
is abbreviation 1 / quadrisection of space depended comparatively for 2 minutes, and 
the reference Fig. heart 2 carries out it together with a division boundary top. 
Distortion evaluation value epsilonk after the first and second convolution 
transformation (k= 1, 2, 3, 4) classifies the obtained second division space S21-S24 
into the division space S21 and S24 settled in the assignment allowable-error value 
delta, and the division space S22 and S23 in which it does not fit. Distortion 



amendment is possible by affine transformation child A (2) in the division space S21 
and S24 settled in the allowable-error value delta. To the division space S22 and S23 
which is not settled in the allowable-error value delta, the still more nearly same 
division is repeated below. Generally by the above, space Sx1 x2 and — which were 
divided the n-th order, and affine transformation child A (n) in xn (xk= 1 , 2, 3, 4, k= 1 , 2, 
— , n) become like a degree type. 
[0040] 

[Equation 13] 
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[0041] Distortion is made to cancel by making this affine transformation child A (n) act. 
By stepping on the above step [ the 1st to 3rd ], it becomes possible to store a 
position error in the assignment allowable-error value delta in respect of [ all ] image 
pick-up space. 

[0042] The gestalt of the operation which applied the algorithm concretely mentioned 
above to the flat-surface configuration gaging system is explained. Drawing 4 shows 
the interference fringe metering device by the phase shift method which picturizes an 
interference fringe image using CCD cameras 412a, 412b, and 412c of plurality (in the 
case of drawing three pieces), and measures a flat-surface configuration. The 
principle of such a coincidence metering device of a phase shift interference fringe is 
proposed by JP,2-287107,A etc. A beam is expanded with a lens 402, and the 
coherent light bundle from a laser light source 401 is collimated by the collimate lens 
404 through the non-polarization beam splitter 403, and is irradiated by the reference 
side 405. the reference side 405 — a part — reflection — it is transparency a part 
and the transmitted light flux is further irradiated by the specimen plane 407 through 
the quarter-wave length plate 406. 

[0043] The reflected light of the reference side 405 and a specimen plane 407 is piled 
up as the hands-off flux of light the flux of light and polarization cross at right angles 



with the quarter-wave length plate 406 (namely, 180 phase contrast), and is reflected 
by the beam splitter 406. This hands-off flux of light is the circular polarization of light 
flux of light which has the flat-surface configuration information on a specimen plane 
407 in the form of the difference of the phase of the reflected light (sample light) of a 
specimen plane 407 to the phase of the reflected light (reference beam) from the 
reference side 405. two non-polarization beam splitters 408a and 408b and total 
reflection mirror 408c divide this hands-off flux of light every [ 2 / 1/] one by one — 
having — three spectra — it goes into optical system 40a, 40b, and 40c. 
[0044] a spectrum — for example, quarter-wave length plate 409a for optical system 
40a and 40b to give phase contrast to the three hands-off flux of lights by a unit of 90 
degrees, and 1/2 wavelength-plate 409b — having — moreover — each — a 
spectrum — optical system 40a, 40b, and 40c — each — a spectrum — it has the 
polarizing plates 410a, 410b, and 410c for making phase contrast information on the 
flux of light into interference fringe information on the strength. As for these polarizing 
plates 410a, 410b, and 410c, the polarization direction inclines by a unit of 45 degrees, 
thereby — each — a spectrum — three interference images of sample light and a 
reference beam which carried out the phase shift every [ 90 ] are picturized by CCD 
cameras 412a, 412b, and 412c of optical system 40a, 40b, and 40c. in addition, the 
attenuation plates 41 1a and 41 1b — each — a spectrum — it is inserted in order to 
keep equal the light-receiving reinforcement in optical system 40a, 40b, and 40c. 
[0045] Three interference fringe information that the phases picturized by CCD 
cameras 412a, 412b, and 412c differ is inputted into an image processing system 413, 
is controlled by the control computer 414 and the monitor 415, and computes the 
configuration information on a specimen plane 407 by the predetermined operation. In 
the case of the gestalt of this operation, the interference fringe information II (x y) is 
concretely acquired from three CCD cameras 412a, 412b, and 412c to each location 
(x y) of image pick-up space, respectively, but Since geometric distortion arises to 
image pick-up space by the optical element prefaced by each CCD cameras 412a, 
412b, and 412c and them, the interference fringe information II (x y) on each location 
(x y) of image pick-up space is not necessarily the interference information over the 
same location. Then, it is necessary to cancel a geometric distortion, and to add 
amendment so that the interference fringe information II (x y) on each location (x y) of 
an image pick-up image may turn into interference information over the same location 
of a specimen plane 407. 

[0046] Then, the algorithm of (1) - (3) mentioned above is applied. An affine 
transformation type [ in / for three image pick-up space l= 1 and 2 or 3 division space 
which were obtained / each division space ] is set like the following type as Slk (k= 1, 
— , nl). 
[0047] 

[Equation 14] 



= A 1 



r "> 

x 1 



Y 1 



+ b 1 c k -A 1 k a \ k 



[0048] It sets to several 14 and they are blgk and algk. The original-drawing heart in 
Slk (k= 1, — , nl) and the imprint original-drawing heart are expressed, respectively. 
Next, each coordinate value (XI, Yl) of three image pick-up space I corresponding to 
each coordinate value (x y) of reference space is given by the following formula. 
[0049] 

[Equation 15] 
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[0050] Therefore, the interference information on three image pick-up space I 
corresponding to each location (x y) of a specimen plane 407 is given for the 
interference fringe information II (XI, Yl) in the coordinate value (XI, Yl) calculated in 
the top. The interference fringe information on three image pick-up equipments 
corresponding to the location where a specimen plane is the same (value on the 
strength [ optical ]) which canceled the geometric distortion of a CCD image pick-up 
image as mentioned above is acquired. 
[0051] 

[Effect of the Invention] As stated above, according to this invention, the geometric 
distortion by measured space and CCD image pick-up space can be easily canceled 
within a predetermined allowable-error value, and highly precise-ization of the 
flat-surface shape-measurement Sagitta label location measurement using two or 
more optical system and a corresponding CCD camera etc. can be expected. 
Moreover, since what is necessary is just to make it act on each field in the phase of 
the beginning of processing of that affine transformation child of image information 
whenever it makes the preparations on which a final affine transformation child is 
made to act for every field and obtains image pick-up image information according to 
this invention unless optical system and an image pick-up system are changed, also in 
time, there is little count cost and it ends. 



DESCRIPTION OF DRAWINGS 



[Brief Description of the Drawings] 

[Drawing 1] It is drawing showing the relation between the reference space in this 
invention, and image pick-up space. 

[Drawing 2] It is drawing showing the relation between the affine transformation in this 
invention, and an allowable-error value. 

[Drawing 3 A] It is drawing showing the example of the space division of the reference 
space in this invention. 

[Drawing 3 B] It is drawing showing the example of the space division of image pick-up 
space similarly. 

[Drawing 4] It is drawing showing the interference fringe image image pick-up system 
by which this invention is applied. 
[Description of Notations] 

1 [ — Reference space, 5 / — An image pick-up graphic form, 6 / — The imprint Fig. 
heart, 7 / — The imprint original-drawing heart, 8 / — Image pick-up space. ] — A 
reference graphic form, 2 — The reference Fig. heart, 3 — The reference 
original-drawing heart, 4 
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